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ABSTRACT 


Indian classical music is a rich and intricate tradition that weaves together melodies, rhythms, and 
intricate improvisations, reflecting a profound connection between spirituality and artistic expression. 
This paper explores the development and implementation of an innovative raga based musical chatbot 
that utilizes NLP techniques to generate Indian Classical music tailored to specific moods or 
emotions. This is done by employing sentiment analysis and deep learning algorithms. The system 
can interpret textual descriptions and facial expression of emotions and translate them into melodious 
compositions. Beyond artistic expression, our system also leverages the therapeutic potential of 
music, allowing users to customize their listening experience for stress reduction and potential health 
benefits. This research bridges tradition and technology, offering a unique approach to raga therapy. 
Keywords— Raga therapy, NLP, Chatbot, Sentiment analysis, CNN 


1. Introduction 

In the realm of musical innovation, "Melody Magic" emerges as a pioneering fusion of Indian 
classical music, advanced technology, and user-centric experiences. This paper presents an ingenious 
approach that utilizes Natural Language Processing (NLP) within a chatbot framework to 
dynamically create Indian classical music compositions tailored to users' emotional states. Indian 
classical music, deeply rooted in cultural heritage, offers a diverse array of 72 melakartha ragas, each 
uniquely associated with chakras and nerve centers in music therapy. The intertwining of ragas and 
musical therapy principles lays a foundation for the paper's core concept. NLP algorithms adeptly 
decipher user text inputs along with Convolutional Neural Networks (CNNs) that are harnessed to 
capture the facial expression and emotion, facilitating a seamless transition from linguistic sentiment 
to musical expression. The harmonious integration of Indian classical music, NLP, and CNN 
technology presents an exciting avenue for personalized musical experiences that resonate with users’ 
emotional landscapes, forging a new paradigm where ancient wisdom harmonizes with contemporary 
innovation. 


2. Literature Survey 
Some of the primary features highlighted in the surveyed papers include: 
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Music therapy [1] is thought to have healing benefits on the body and psyche, as evidenced by the 
use of Indian classical ragas to cure ailments. Music's vibrations stimulate the central nervous system 
and can increase blood flow and energy flow in the body, facilitating faster recovery. Overall, music 
therapy is regarded as a highly effective method for relaxation, healing, and improving well-being. 
[2] Music and movie recommendation chatbot determines the mood of the user, followed by music 
that will be played or collection of films that will be displayed on a website depending on the mood. 
The goal is to assess the user's emotional state consistently and accurately. This comprehensive plan 
aims to provide a more engaging and individualized entertainment experience. 

Therapeutic [3] use of Indian ragas to various disorders investigates how Indian ragas can be used 
to treat a range of diseases. The study employs EEG to measure brainwave patterns and assess the 
effects of listening to specific ragas. Electrodes attached to the scalp capture data, which are then 
converted to digital signals for examination. More empirical data and research are needed to fully 
understand music therapy's therapeutic potential. [4 The Music Recommender System Chatbot is 
designed to suggest music based on the tone or context of the user's text. The system makes use of an 
IBM service to precisely estimate the user's mood. Once the mood is identified, the chatbot will 
provide personalised song recommendations on a web page. 


Raga Therapy [5] An Effective Stress Management Treatment. The study included 74 students, both 
male and female, with moderate to high levels of stress. A survey was employed to evaluate the stress 
levels of individuals both prior to and following a 30-minute raga therapy intervention. The outcomes 
of the intervention demonstrated a considerable reduction in stress levels. According to the study, 
raga therapy can be a non-invasive, low-cost stress management strategy. Natural Language 
Processing [6] The approach for the Musical Instruments Recommendation System centres around 
creating a system for recommending musical instruments. This system aims to provide the user with 
the top three instrument recommendations based on their search query. This search query provides 
technical details, descriptions, sentiments, and brand information about musical instruments. 


Study of Indian Classical Ragas Structure [7] and its Influence on Human Body for Music Therapy, 

focuses on the emotions elicited when listening to various ragas. A database of EEG signals from 
participants who listened to various ragas will be compiled for the study, and the brainwave patterns 
will be examined in relation to various emotions. The purpose of the study is to support the therapeutic 
benefits of music and aid medical professionals in using music as a treatment stimulus for their 
patients. In an examination of a Music Recommendation System [8] that considers facial expressions, 
whether the user is wearing a face mask or not, the paper discusses a system for recommending music. 
This system utilizes facial expressions to gauge a user's mood and musical preferences, even when 
the user has a face mask on. The proposed method modifies existing CNN architectures by 
incorporating specialized layers that specifically analyse visible facial areas like the eyes and 
eyebrows. 
An effect of Raga Therapy [9] on our human body, demonstrated that music therapy helps people 
with neurological disorders, mental and physical disabilities, pain, anxiety, and depression. Ragas 
can activate chakras, and the seven swarms correspond to the chakras. For optimum results, ragas are 
used in conjunction with ayurveda. Overall, there are many potential mental health advantages to 
music therapy. Deep learning algorithms [10] and natural language processing techniques are used 
in the music recommendation bot. By evaluating the user's conversation and mood, the bot is intended 
to create a personalised music experience. The chatbot mimics human interaction, while the emotion 
detection module ascertains the user's mood. The music suggestion module provides tracks based on 
the user's mood. 


The method is based on [11] facial recognition to identify students' understanding of the distance 
learning process. The method consists of feature extraction, subset feature, and emotion classification. 
The proposed method shows high classification performance in recognizing six different emotional 
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categories. A real time face emotion classification [12] and recognition using deep learning model. 
The system utilizes convolutional neural networks (CNN) for face detection, recognition, and 
emotion classification. The proposed system is trained on a large database and is capable of 
classifying emotions such as anger, sadness, happiness, fear, disgust, and surprise. 


Optimisation of emotion detection [13] machine learning algorithm in terms of human facial 
expressions focuses on optimising the recognition method of seven basic emotions based using 
the neural network with an optimised algorithm. They reach 72% accuracy and do a computational 
experiment to evaluate the model's efficiency on a larger collection of training photos. A unique facial 
expression analysis [14] approach investigates the use of facial expression analysis to assess the 
effectiveness of high and low emotional adverts on individual driving attitudes. According to the 
study, high emotional commercials resulted in a lower decline in the speed limit reinforcement score. 
[15] The document highlights the potential of this approach in intersecting big data, machine learning, 
and facial recognition for various applications. The proposed big data architecture integrates facial 
recognition and sentiment analysis, providing valuable insights into people's mood and sentiments. 


3. Methodology 

3.1 Melody Magic 

The following data flow diagram illustrates the seamless interactions and operations of our Chatbot, 
shedding light on its functionalities and how it aids in providing efficient and responsive user 
experiences. 


Chatbot using Type of 


NLP input Face Detection 


10 Sentiment Emotion detection 
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Recommend ( Generate raga 
raga/songs to () based classica 
the user muisc 


Fig. 1. Comprehensive Insights into Chatbot Functionality 
Through extensive research and a comprehensive survey involving over 50 individuals, it became 
evident that a significant majority of people were grappling with compromised mental and emotional 
well-being. The survey revealed prevailing feelings of stress, depression, and emotional instability 
within this demographic. In response to these pressing concerns, Melody Magic" chatbot was 
developed. 


"Melody Magic" serves as a Chatbot at its core, but its functionalities extend far beyond. Enriched 
with a robust emotion detection and Sentiment Analysis module, this application stands as a beacon 
of hope for those in need. Its primary objective is to provide solace and tranquillity to users by 
harnessing the power of Indian classical music and ragas tailored to their prevailing emotional states. 
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Figure 1 elegantly illustrates the inner workings of this application. Upon visiting the website, users 
are warmly greeted by the "Melody Magic" chatbot window, which serves as the primary gateway to 
our transformative technology. This creation is a direct response to the mental health challenges 
uncovered during our survey, as it seeks to alleviate emotional burdens and enhance the overall well- 
being of individuals through music and thoughtful interaction. 


In the first option, users can communicate with the chatbot by typing their messages into a text field. 
The chatbot recommends a raga after assessing the user's mood. The second interaction method is 
initiated by clicking the camera button adjacent to the text box. This method leverages image analysis 
to recognize the user's mood and subsequently offers raga or song recommendations based on the 
detected emotion. 


Table.1. Raga classification according to mood 


Raga Emotion/Mood 


Raga Bhairavi Emotional strength, Devotion and Peace 


Raga Gunji Kanada, Raga Neelambari, Raga 


Bihag, Raga Jaijawanti PoPa 


Raga Lalita, Raga Abhogi Peace & Tranquillity 


Raga Todi, Raga Hamsadwani, Raga Toy. Pappie 


Madhuvanti 

Raga Brindavani Saranga, Raga Desh Greater Energy 

Raga Puriya, Rageshri, Purvi Kalyani Harmony and Rejuvenation 
Raga Kapi Depression, Anxiety 

Raga Bilahari Calm, Release Anger 

Raga Suruti Fear 

Raga Shuddha Saranga Victory, Success 

Raga Yamuna Kalyani Freshness, Energy booster 


3.2 Facial Detection and Emotion Recognition 

We gathered a diverse dataset of facial images encompassing a broad spectrum of emotions, including 
joy, sadness, anger, and more, from publicly available sources and labelled them accordingly. The 
collected images underwent preprocessing to enhance quality and standardize features, including 
resizing to a consistent resolution and grayscale conversion. We further employed a pre-trained CNN 
architecture, specifically designed for facial recognition and emotion classification, as our primary 
model for facial detection and emotion recognition. Transfer learning was used, fine-tuning the pre- 
trained model on our emotion-labeled dataset. The dataset was split into training and validation sets, 
and the CNN model was trained using backpropagation and optimized using appropriate loss 
functions and evaluation metrics. 
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3.3 Text Emotion Identification 

A corpus of text data containing user inputs was collected, spanning a range of conversational 
contexts and emotional expressions. The text data was pre-processed, including tokenization, stop- 
word removal, and stemming, to prepare it for emotion analysis. We utilized an emotion lexicon to 
associate words and phrases with specific emotional categories. We employed a machine learning 
classifier, specifically trained for text emotion identification, using techniques such as Bag-of-Words 
(BoW) and Term Frequency-Inverse Document Frequency (TF-IDF) vectorization. 


3.4 Chatbot Construction Using NLP 

To build the chatbot, we utilized a state-of-the-art Natural Language Processing (NLP) framework, 
which enabled us to process and understand user inputs in natural language. Combining the results 
from both the facial detection and emotion recognition CNN model and the text emotion identification 
NLP model, the chatbot inferred the user's mood based on textual and visual cues. The chatbot 
recommended Indian classical ragas or music compositions tailored to the user's mood, drawing from 
a curated database of compositions categorized by emotional resonance. 


3.5 The Raga Therapy 

Ragas in Indian classical music are not only appreciated for their artistic and aesthetic qualities but 
also for their potential therapeutic benefits. Ragas are known to evoke specific emotions and moods 
through their unique melodic and rhythmic structures. Different ragas are linked to various emotions, 
including calm, joy, love, and grief. Certain ragas' have contemplative and relaxing properties that 
can help people feel less stressed and promotes inner serenity and provides relaxation. Certain ragas 
are thought to boost brain activity and improve cognitive abilities. Ragas can be used by traditional 
Indian musicians and music therapists to assist people deal with problems including anxiety, sadness, 
and chronic pain, stress, anxiety. While there is anecdotal evidence and some continuing research into 
the therapeutic advantages of ragas, it's crucial to emphasise that the scientific validity of these claims 
is still an area of inquiry and discussion. The effectiveness of music therapy, including Indian 
classical music therapy, can vary from person to person, and it should not be considered a replacement 
for conventional medical treatment when needed. 


Result 


Users reported a high level of satisfaction with the chatbot's ability to recommend Indian classical 
music or ragas aligned with their emotional states. The generated music was found to be emotionally 
impactful, contributing to feelings of calmness, upliftment, or introspection, based on the detected 
mood. In a survey involving 100 participants, we found that 80 people had stress and mental illness 
and 75 people response reported shows an uplift in their emotional state after using the system. This 
substantial accuracy in mood enhancement underscores the system's potential as a valuable tool for 
improving mental well-being through the therapeutic power of music. Further research and 
development in this domain could refine the system for even higher accuracy, offering a promising 
avenue for technology-driven mental health support. 


CONCLUSION 

This research carries substantial real-world significance as it presents a technology-driven solution 
that has the potential to enhance users' emotional well-being by uniting Natural Language Processing 
(NLP), Convolutional Neural Networks (CNN), and music recommendation. As we move forward, 
further research and development in this domain hold the promise of even higher accuracy, an 
expanded music database, and continued advancements in the intersection of technology, emotion, 
and well-being. This raga based chatbot has not only showcased the efficacy of AI in the realm of 
emotional support but also sets the stage for a harmonious future where technology and empathy 
intertwine for the betterment of human experiences. 
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